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Abstract 

The  modern  warfighter  operates  in  an  environment  that  has  dramatically  evolved  in 
sophistication  and  interconnectedness  over  the  past  half  century.  With  each  passing  year,  the 
infusion  of  ever  more  complex  technologies  and  integrated  systems  places  increasing 
burdens  on  acquisition  officers  to  make  decisions  regarding  potential  programs  with  respect 
to  the  joint  capability  portfolio.  Furthermore,  significant  cost  overruns  in  recent  acquisition 
programs  reveal  that,  despite  efforts  since  2010  to  ensure  the  affordability  of  systems, 
additional  work  is  needed  to  develop  enhanced  approaches  and  methods.  This  paper 
discusses  research  that  builds  on  prior  work  that  explored  system  design  tradespaces  for 
affordability  under  uncertainty,  extending  it  to  the  program  and  portfolio  level.  Time-varying 
exogenous  factors,  such  as  resource  availability,  stakeholder  needs,  or  production  delays, 
may  influence  the  potential  for  value  contribution  by  constituent  systems  over  the  lifecycle  of 
a  portfolio,  and  make  an  initially  attractive  design  less  attractive  over  time.  This  paper 
introduces  a  method  to  conduct  portfolio  design  for  affordability  by  augmenting  Epoch-Era 
Analysis  with  aspects  of  Modern  Portfolio  Theory.  The  method  is  demonstrated  through  the 
design  of  a  carrier  strike  group  portfolio  involving  the  integration  of  multiple  legacy  systems 
with  the  acquisition  of  new  vessels. 

Introduction 

Enabled  by  the  emerging  widespread  availability  of  high  speed  computing, 
computational  Tradespace  Exploration  (TSE)  has  become  a  valuable  tool.  TSE  empowers 
system  engineers  to  consider  far  more  potential  designs  than  could  be  done  through  prior 
Analysis  of  Alternatives  methods  (Ross  &  Hastings,  2005).  A  recently  developed  approach, 
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Epoch-Era  Analysis  (EEA),  enables  the  conceptual  design  of  systems  that  are  resilient  to 
potential  changes  in  context  and  needs  (exogenous  uncertainties)  over  the  lifecycle  of  the 
system.  EEA  enables  quantitative  support  for  the  design  of  particular  time-contingent 
system  properties  such  as  survivability,  flexibility,  and  affordability  (Ross  et  al.,  2010). 

Since  the  issuance  of  the  2010  memo  Better  Buying  Power  (BBP)  and  the 
implementation  of  the  Department  of  Defense  (DoD)  Efficiency  Initiative  (Carter,  2010),  a 
significant  body  of  research  has  been  generated  to  support  design  for  affordability,  including 
advanced  TSE  techniques.  To  this  end,  EEA  was  adapted  for  affordability  analysis  in  naval 
acquisitions  by  Schaffner  et  al.  (2013)  through  the  introduction  of  aggregate  cost  and 
schedule  considerations.  As  system  interconnectedness  and  interdependence  continues  to 
increase,  especially  in  Navy  and  DoD  operations  involving  numerous  assets,  conceptual 
design  techniques  that  consider  only  the  acquisition  and  operation  of  individual  systems  are 
not  fully  sufficient.  Of  additional  value  in  this  regard  is  expanding  the  scope  of  such 
techniques  to  support  acquisition  decisions  at  the  multi-system  level;  this  necessitates 
consideration  of  two  related  concepts:  systems  of  systems  (SoS)  and  portfolios. 

A  SoS  is  a  dynamic  network  of  constituent  systems  that  exhibit  varying  levels  of 
operational  and  managerial  independence,  but  operationally  interact  so  as  to  achieve 
mutually  desired,  oftentimes  emergent,  capabilities  (Maier,  1999).  A  portfolio  is  a  construct 
which  describes  a  collection  of  assets,  acquisition  programs,  and  research  programs  that 
are  jointly  invested  in  to  exploit  qualities  of  the  set,  regardless  of  whether  the  assets  are 
operationalized  independently  or  participate  in  a  SoS.  The  Carrier  Strike  Group  (CSG)  case 
study  conveniently  illustrates  the  difference  between  SoS  and  portfolio  design.  The  CSG 
portfolio  design  problem  seeks  to  ascertain  what  acquisition  strategies  result  in  an 
affordable  set  of  available  assets  that  may  be  assembled  into  any  number  of  CSG  SoS  to 
meet  a  variety  of  possible  desired  performance  attributes.  The  SoS  design  problem,  on  the 
other  hand,  would  occur  downstream  of  portfolio  design  where  a  designer  seeks  to  select 
available  assets  from  the  portfolio,  apply  concept  of  operations  (CONOPS)  to  dictate  SoS 
interactions,  and  meet  specific  desired  capabilities. 

SoS  and  portfolio  design  considerations  represent  unique  challenges  for  tradespace 
analysis,  and  especially  for  affordability  analysis  under  uncertainty.  Because  engineering 
portfolios  may  include  assets  that  are  SoS  themselves,  and  because  there  is  typically  a  high 
degree  of  interaction  and  interdependencies  in  the  costs,  risks,  and  capabilities  of  the 
assets,  traditional  portfolio  assessment  techniques  must  be  modified  to  address  these 
complexities  that  violate  prior  assumptions.  SoS-based  design  approaches  enable 
consideration  of  these  unique  qualities  of  engineering  portfolios  and  shall  inform  the 
development  of  the  method  presented  in  this  research. 

This  paper  discusses  recent  efforts  to  adapt  a  resource-centric  approach  to  EEA 
developed  by  Schaffner,  Ross,  and  Rhodes  (2014)  for  use  in  affordable  portfolio  design 
through  the  integration  of  elements  of  Modern  Portfolio  Theory  (MPT)  and  the  SoS  design 
literature.  The  proposed  method  is  demonstrated  in  a  case  study  on  the  acquisition  of  a 
portfolio  of  assets  from  which  a  CSG  may  be  assembled.  The  method  is  leveraged  to 
provide  design  insight  into  CSG  lifecycle  affordability.  This  insight  is  achieved  by  identifying 
the  utility  and  costs  of  portfolio  designs  in  potential  future  contexts  that  embody  a  variety  of 
uncertainty  factors,  such  as  unit  availability,  budget  constraints,  capability  requirements, 
strategic  threats,  and  technology  development.  The  1 1  processes  of  the  proposed  method 
are  described  as  applied  to  the  CSG  portfolio,  and  they  illustrate  the  method’s  potential 
value  for  naval  acquisition  and  operations.  Furthermore,  a  discussion  of  the  potential  for 
broad  applicability  of  the  method  to  other  DoD  capability  portfolios  is  proffered,  specifically 
with  respect  to  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS). 
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Motivation 

Between  1997  and  201 1 ,  there  were  74  Nunn-McCurdy  program  unit  cost  breeches 
in  47  of  134  major  DoD  acquisition  efforts  (Government  Accountability  Office  [GAO],  2011). 
According  to  an  audit  by  the  GAO,  many  of  these  breeches  corresponded  to  context 
changes  in  the  environment  surrounding  the  acquisition  programs.  These  context  changes 
included  affordability  measurement  statute  modifications,  presidential  administrations 
turnover,  unit  order  size  reduction,  schedule  changes,  and  requirements  changes.  These 
cost  breeches  occurred  in  over  one  third  of  major  DoD  acquisition  programs  and  indicate  the 
need  for  conceptual  design  methodologies  that  consider  potential  changes  in  context  in 
order  to  achieve  consistent  lifecycle  program  affordability. 

In  light  of  these  breeches,  and  supported  by  the  DoD  emphasis  of  an  “affordability 
mandate”  in  BBP  1.0,  2.0,  and  now  3.0  (Carter,  2010;  Kendall,  2012,  2014),  design  for 
affordability  literature  and  practice  grew  rapidly.  This  provides  the  foundation  for  extending 
EEA  to  the  portfolio-level.  First,  the  concept  and  meaning  of  the  term  affordability  was 
explored  and  defined  by  many  organizations  including  INCOSE  and  the  NDIA,  as  well  as  by 
the  DoD  (Schaffner  et  al.,  2013).  Second,  metrics  were  developed  to  assess  the  contextual 
and  dynamic  attributes  of  affordable  systems  (Bobinis  et  al.,  2013),  and  a  variety  of  tools 
were  produced  to  support  affordability  tradeoffs  between  potential  systems.  Third,  EEA  was 
employed  to  reveal  system  affordability  across  a  variety  of  uncertain  futures  and  to  provide 
insight  into  changing  contexts,  such  as  those  which  caused  many  of  the  recent  Nunn- 
McCurdy  breeches  (Wu,  Ross,  &  Rhodes,  2014). 

These  advances,  however,  address  only  a  part  of  the  challenge  in  achieving 
affordability  for  the  DoD  since  they  do  not  explicitly  consider  the  higher  order  complexities 
inherent  to  multi-system  acquisition  and  operations  (Wu,  2014).  As  early  as  2010,  DoD 
decision  makers  recognized  that  design  for  affordability  at  the  system  level  did  not 
necessarily  translate  to  the  affordability  of  the  overall  capability  portfolio.  Remarks  by 
General  Peter  Chiarelli  indicate  an  understanding  of  this  concept  and  call  for  a  portfolio-level 
conceptual  design  paradigm  (Association  of  the  United  States  Army,  2010,  p.  1): 

If  you  look  at  any  one  of  these  systems  as  an  individual  system,  you  can  sell 
just  about  anything.  But,  when  you  look  at  [an]  entire  portfolio  you  can  start  to 
see  where  we  have  duplication  in  different  systems  or  maybe  we’re 
overinvesting  in  one  and  underinvesting  in  another. 

A  variety  of  portfolio  management  techniques  have  been  developed  to  begin  to  fill 
this  design  and  acquisition  capability  gap.  The  Systems  Engineering  Guide  for  System-of- 
Systems  (2008)  provides  guidelines  for  the  engineering  of  SoS  in  DoD  acquisitions. 
Komoroski  et  al.  (2006)  applied  a  variant  of  real  options  analysis  to  identify  long-term  SoS 
acquisition  strategies  for  information  technology.  The  Computational  Exploratory  Model  by 
Mane  and  DeLaurentis  (201 1)  was  developed  to  assess  development  networks  of  SoS 
architectures.  Initial  efforts  to  inform  the  acquisition  and  integration  of  systems  in  a  SoS 
through  MPT  were  also  made  by  Davendralingam,  Mane,  and  DeLaurentis  (2012).  Epoch- 
Era  Analysis  complements  these  previous  efforts  by  adding  the  ability  to  assess  the 
influence  of  changing  contexts  on  the  affordability  of  potential  portfolios. 

Extending  EEA  for  Affordability  From  “System-Level”  to  “Portfolio-Level” 

To  clarify  the  terminology  used  in  this  paper  and  link  it  to  trends  in  the  SoS  literature, 
a  set  of  terms  is  introduced  to  describe  three  “tiers”  of  design  abstraction.  Table  1  presents 
an  example  of  the  design  scope  for  each  tier  of  design  abstraction  as  applied  to  the  CSG 
case. 
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System-Level:  System  design  occurs  at  a  “level  of  decomposition  that  is  inclusive  of 
a  major  architectural  element  and  is  semi-independent  from  the  rest  of  the  architecture” 
(Walton,  2002).  The  designer  of  a  system  typically  has  full  design  authority,  and  although  a 
system  may  be  composed  of  multiple  components  (such  as  a  launch  vehicle),  it  is  not 
considered  a  SoS  as  the  sub-elements  are  not  managerially  independent  (Maier,  1999). 

Program-Level:  Program  design  is  distinguished  from  system  design  in  that  it 
requires  joint  consideration  of  multiple  independent  or  semi-independent  constituent 
elements  (typically  systems  themselves).  However,  like  system-level  analysis,  the  designer 
of  a  program  is  typically  assumed  to  have  a  moderate  to  high  degree  of  design  authority. 
Two  primary  types  of  programs  have  been  identified,  and  they  are  distinguished  by  the 
attributes  of  the  constituent  systems: 

•  Type  I  programs  are  composed  of  homogenous  constituent  systems.  Type  I 
program  design  is  readily  conducted  through  traditional  EEA  where  the  most 
promising  single  system  solution  is  also  expected  to  produce  the  greatest 
program  benefit.  Initial  research  into  Type  I  programs  was  conducted  by  Wu 
(2014). 

•  Type  II  programs  are  composed  of  heterogeneous  constituent  systems  which 
often  complete  similar  missions  and  are  evaluated  by  a  common  set  of  value 
metrics.  Because  Type  II  programs  concern  either  semi-independent  or  fully- 
independent  constituent  systems,  as  opposed  to  the  closely  managed 
homogenous  systems  in  Type  I  programs,  the  design  problem  involves  SoS 
challenges. 

Portfolio-Level:  A  portfolio  is  a  collection  of  selected  assets  that  may  be  either  new 
or  legacy  programs  as  defined  above,  which  are  simultaneously  invested  in  to  collectively 
provide  a  set  of  capabilities.  A  portfolio  designer  does  not  necessarily  have  a  significant 
level  of  control  over  the  design  of  the  constituent  programs,  or  their  ultimate 
operationalization  in  a  SoS,  but  can  create  a  portfolio  with  attractive  procurement, 
management,  and  capability  features  based  upon  the  possible  assets  and  their  likely 
applications.  Design  at  the  portfolio-level  must  not  only  consider  traditional  financial  portfolio 
investment  techniques  that  identify  emergent  properties  from  a  set  of  independent  assets, 
but  also  must  consider  SoS  techniques  that  consider  value  arising  from  the  potential 
interaction  of  the  assets  when  operationalized. 

Table  1 .  Design  Abstraction  Tiers  as  Applied  to  a  Carrier  Strike  Group 


SYSTEM 

PROGRAM 

Type  /  Type  U 

PORTFOLIO 

Basic 

Unit 

Arieiglr  Burke-class 
destroyer 

Arleigh  Burke -cl ass 
destroyer  fleet 

Carrier  strike  group 
guided  missile  support 

Carrier  strike  group 

Design 

Goal 

Meet  required  capability 
criteria  for  assigned 
mission 

Provide  multi-mission 
offensive  and  defensive 
capabilities 

Provide  multi-mission 
guided  missile  support 
to  a  potential  CSG 

Provide  military 
dominance  in  assigned 
sphere  of  influence 

Sub- 

Units 

Destroyer  components: 

*  GE  LM  2500  Engines 

-  Superstructure 

-  Weaponry 

-  Crew  life  support 

-  Etc. 

-  DDG-51 

-  DDG-52 

-  DDG-112 

-  Arleigh  Burke-class 

-  Zum wait-class 

-  Tic  onderoga-c lass 

-  Virginia-class 

-  Los  Angles-class 

-  Nimitz-class  carrier 
*  Cruisers 

-  Destroyers 

-  Attack  submarines 

-  Aircraft  complement 

-  Etc, 

In  prior  research,  Wu  (2014)  proposed  a  methodology  for  program  and  portfolio-level 
affordability  analysis  utilizing  a  bottom-up  “survival  of  the  fittest”  approach.  Wu’s  approach 
leveraged  EEA  to  identify  promising  individual  systems  for  affordability;  such  system-level 
analysis  is  an  extensively  explored  capability  of  EEA.  At  the  next  tier  of  design  abstraction, 
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the  program  level,  new  program-level  performance  attributes  were  applied  to  a  set  of 
programs  seeded  from  the  “promising”  systems  identified  in  system-level  design.  It  was  then 
ascertained  which  combination  of  these  systems  most  promisingly  fulfilled  the  program-level 
goals.  This  “wrapping”  approach  was  a  logical  first  step  towards  program  and  portfolio 
design  with  EEA.  However,  the  method  may  prematurely  discard  potentially  valuable 
designs  that  do  not  appear  attractive  in  lower  levels  of  design  abstraction,  but  produce  a 
favorable  overall  portfolio. 

The  concern  for  premature  abandonment  of  potential  assets  is  supported  by  Walton 
(2002).  Walton’s  work  applied  portfolio  theory  to  tradespace  analysis  for  a  space  science 
mission  and  found  that  the  most  promising  portfolio  of  assets  to  minimize  uncertainty  was 
not  consistently  a  constellation  of  the  most  promising  single  satellite  design(s).  Rather, 
Walton  found  that  designs  for  portfolio-level  minimum  uncertainty  included  sub-par,  system- 
level  solutions  that  interacted  to  exhibit  emergent  benefits  and  the  highest  portfolio-level 
utility. 

Recognizing  the  limitations  of  both  the  bottom-up  and  top-down  (traditional 
requirements-driven  design)  approaches  to  portfolio  design  for  affordability,  Wu  suggested 
that  a  two  pronged  method  to  leverage  the  strengths  of  each  approach  may  lead  to  higher- 
utility  portfolios.  TSE  is  inherently  a  bottom-up  process  where  constituent  systems  are  first 
designed  for  desired  system  level  attributes,  and  then  linked  together  through  multiple  levels 
of  larger  subsystems  to  result  in  portfolio  capabilities.  MPT  is  an  inherently  top-down 
approach  where  designers  identify  desired  portfolio-level  attributes  (big  picture),  and  an 
algorithm  then  seeks  to  compile  a  set  of  investments  (systems)  which  satisfy  those  attributes 
in  aggregate  (Amenc  &  Sourd,  2005).  This  paper  intends  to  leverage  both  of  these 
techniques  by  applying  elements  of  MPT  and  TSE  to  EEA.  Synthesized  bottom-up  and  top- 
down  models  have  found  traction  in  land  use  portfolio  planning  and  energy  portfolio 
management  (Castella  et  al.,  2007;  Wing,  2006). 

Overview  of  an  EEA-Based  Approach  for  Design  for  Affordability 

Acquisition  program  planning  under  affordability  considerations  requires  analysis  of 
factors  such  as  system  development  schedules,  legacy  hardware  and  operations, 
resources,  and  political  capital.  Furthermore,  these  factors  must  be  considered  with  respect 
to  the  dynamic  environments  in  which  they  exist  over  the  system  lifecycle.  Epoch-Era 
Analysis  is  an  effective  mechanism  to  evaluate  acquisition  strategies  in  anticipation  of  future 
context  shifts.  An  epoch  is  a  time  period  of  static  context  and  stakeholder  expectations,  like 
a  snapshot  of  a  potential  future.  An  era  is  an  ordered  sequence  of  epochs  with  finite 
durations  that  describe  a  potential  progression  of  contexts  over  the  lifecycle  of  the  system. 

EEA  consists  of  several  distinct,  but  related  analysis  techniques.  In  single-epoch 
analysis,  potential  portfolios  are  evaluated  in  individual  epochs  to  determine  how  close  to 
the  Pareto  front  the  portfolio  lies.  When  the  performance  of  the  same  portfolio  is  compared 
in  different  epochs  through  multi-epoch  analysis,  the  various  influences  of  contextual 
uncertainties  may  be  perceived  and  shifts  in  portfolio  proximity  to  the  Pareto  front  may  be 
observed.  If  ordered  epochs  are  strung  together  into  an  era,  changes  in  the  value  of 
proposed  portfolios  over  time  becomes  apparent,  and  portfolios  which  may  become 
unaffordable  are  identifiable.  Figure  1  illustrates  how  single-era  analysis  may  reveal  lifecycle 
deficiencies  in  initially  promising  portfolio  designs. 
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Note.  This  Epoch-Era  Analysis  reveals  the  performance  (including  affordability)  of  a  system  through  a  sequence 
of  varying  contexts  illustrating  the  potential  lifecycle  value  of  the  system. 


Figure  1.  Epoch-Era  Analysis 

(Ross  &  Rhodes,  2008) 

In  prior  research,  Schaffner  et  al.  (2014)  developed  a  composite  method  for 
affordability  based  upon  the  Responsive  Systems  Comparison  (RSC)  method,  and  applied 
their  approach  to  the  acquisition  of  a  Next  Generation  Combat  Ship  (NGCS)  for  the  Navy. 
RSC  is  a  variant  of  EEA  which  leverages  TSE  as  a  powerful  tool  to  conduct  analysis. 
Complex  system  design,  and  particularly  SoS  design,  involves  multiple  dimensions  of 
relevant  benefits,  expenses,  and  boundary  conditions  which  do  not  lend  themselves  to 
optimization  and  are  often  too  complex  for  intuitive  decision  making.  TSE  applies  the 
capabilities  of  modern  computing  to  enumerate  a  large  variety  of  design  alternatives  to 
support  a  decision  maker  to  holistically  investigate  subtle  tradeoffs  and  previously 
unconsidered  designs.  As  a  result,  RSC  may  lead  a  designer  to  select  an  acquisition 
program  with  superior  lifecycle  results  than  those  determined  by  a  numerical  optimizer 
operating  under  simple  tradeoffs  and  rules  of  thumb  (Wu  et  al.,  2014).  Building  on  this  prior 
research,  this  paper  describes  the  extension  of  the  method  to  the  program  and  portfolio 
levels. 


Overview  of  Modem  Portfolio  Theory  in  Systems  Engineering 

Modern  Portfolio  Theory  (MPT)  has  a  ubiquitous  impact  on  financial  investment 
strategies.  MPT  provides  a  methodology  to  identify  an  efficient  frontier  of  portfolio 
investments  based  upon  the  elicited  values  and  preferences  of  the  investor(s)  and  the  asset 
performance  forecasts.  Such  an  efficient  frontier  is  composed  of  potential  portfolios  of 
investments  which  maximize  the  return  on  investment  while  minimizing  the  risk.  To  achieve 
this  end,  MPT  relies  on  the  concept  that  groups  of  investments  with  negative  trending 
covariance  exhibit  a  portfolio  risk  which  is  less  than  the  average  of  the  risks  of  each 
constituent  investment  (Amenc  &  Sourd,  2005). 

Numerous  derivatives  have  extended  the  capabilities  of  MPT.  Post-Modern  Portfolio 
Theory  (PMPT)  allows  for  the  consideration  of  non-normally  distributed  risks  (a  more 
realistic  assumption)  and  provides  for  the  minimization  of  “downside  risk”  (negative 
outcomes)  rather  than  mean  variance  (Swisher  &  Kasten,  2005).  Walton  (2002)  employed 
TSE  to  allow  for  formal  tradeoffs  between  value  and  uncertainty  (risk)  in  an  effort  to  reveal 
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“synergistic  combinations  of  architectures.”  Furthermore,  Walton’s  work  introduced  semi¬ 
variance  as  a  method  to  treat  upside  and  downside  risks  independently  in  portfolio 
optimization.  The  authors  have  elected  to  utilize  MPT  in  this  initial  application  to  EEA  due  to 
its  greater  simplicity,  and  the  availability  of  literature  documenting  application  to  fields 
beyond  finance. 

The  fusion  of  aspects  of  portfolio  theory  with  TSE  is  not  a  novel  concept. 
Davendralingam  et  al.  (2012,  p.  63)  introduced  a  modification  to  a  model  of  SoS  acquisition 
originally  developed  by  Lane  et  al.  (2010)  as  a  mechanism  to  support  portfolio  design  and 
“maintain  compliance  with  the  ‘top-down  integration,  bottoms-up  implementation’  paradigm” 
of  traditional  SoS  design.  Davendralingam  et  al.’s  approach  relied  upon  an  ongoing,  iterative 
circuit  of  design,  implementation,  and  feedback  to  mitigate  dynamic  contextual  uncertainties. 
The  method  in  this  paper  utilizes  EEA  to  consider  the  potential  uncertainties  of  the  internal 
and  external  environment  at  the  outset,  and  select  a  portfolio  which  is  resilient  against 
changes  in  context. 

Generalization  of  MPT  and  Combination  with  EEA 

While  MPT  and  EEA  share  a  variety  of  commonalities  including  value  elicitation  from 
stakeholders/investors,  the  use  of  models  to  describe  investment/system  value,  and  a 
foundation  in  utility  theory,  there  are  also  fundamental  differences  in  the  design  of  SoS 
which  violate  assumptions  necessary  for  MPT.  Ricci  and  Ross  (2012)  conducted  a  review  of 
the  similarities  and  differences  of  MPT  and  EEA.  Their  work  yielded  Table  2,  which 
describes  key  differences  between  financial  and  SoS  portfolios. 


Table  2.  List  of  Salient  Differences  Between  Financial  and  SoS  Portfolios 

(Ricci  &  Ross,  2012) 


Financial  Portfolios 

SoS  Portfolios 

Often  assume  normally  distributed  asset  returns 

Performance  models  use  various  distributions 

Linear  aggregation  of  asset  return 

Non-linear  relationships  dictate  asset  aggregate 
performa  nee 

Covariance  can  express  correlation  of  assets 

Covariance  is  not  sufficient  to  describe  correlation 

Various  types  of  assets  are  often  generally  available 

Assets  may  be  limited  based  on  economics,  technology, 
politics 

Availability  of  assets  remains  fairly  constant 

Assets  may  change  availability  over  time 

No  portfolio  diversification  cost 

Diversification  costs 

Low  carrying  costs 

Possibly  large  carrying  costs 

Small  switching  costs 

Possibly  large  switching  costs 

Ricci  and  Ross  propose  a  variety  of  modifications  to  MPT  which  address  asset 
availability,  diversification  costs,  carrying  costs,  and  switching  costs.  However,  the 
disconnect  between  SoS  and  financial  portfolios  over  non-normal  distributions  and  non¬ 
linear  relationships  of  constituent  systems  to  portfolio  value  is  yet  to  be  addressed. 

In  MPT,  the  distribution  of  return  (utility)  is  assumed  to  be  a  normal  distribution 
around  the  expected  value.  This  is  an  effort  of  MPT  to  characterize  asset  response  to 
potential  changes  in  future  context.  The  assumption  of  a  normal  distribution  may  not  be 
appropriate  for  engineering  portfolios  where,  especially  for  novel  systems,  thee  is  not  a  large 
set  of  historical  data  upon  which  constituent  system  performance  simulations  may  be 
grounded.  EEA  proves  a  convenient  mechanism  to  address  this  challenge.  Multi-Epoch  and 
era-level  analyses  reveal  changes  in  utility  through  possible  changes  in  context  and 
expectations,  while  also  identifying  path-dependent  uncertainty.  EEA  may  therefore  be  used 
to  determine  promising  portfolios  that  maintain  acceptable  value  across  these  potential 
futures,  rather  than  relying  on  value  predictions  from  the  aggregation  of  utility  distributions. 

In  a  sense,  when  EEA  and  MPT  are  combined,  assumptions  about  the  distribution  of  risk  of 
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constituent  systems  no  longer  need  to  be  made,  but  rather  the  impact  of  various  uncertainty 
factors  may  be  readily  simulated  and  examined  in  the  ultimate  design  choices. 

When  considering  an  engineering  portfolio,  the  utility  (and  expense)  of  the  overall 
portfolio  may  not  simply  be  the  aggregated,  linear-sum  of  the  stand-alone  utilities  of  the 
constituent  elements.  Rather,  engineering  portfolio  utility  may  be  greater  or  less  depending 
upon  additional  operational  relationships  among  the  constituent  elements.  For  example, 
while  a  Ticonderoga-class  cruiser  provides  a  great  deal  of  anti-ship  missile  defense  value, 
two  cruisers  do  not  necessarily  provide  a  potential  CSG  with  twice  the  value.  The  anti¬ 
missile  systems,  radar  systems,  and  concept  of  operations  (CONOPS)  used  by  the  cruisers 
are  identical  (and  therefore  susceptible  to  the  same  vulnerabilities)  and  would  not  provide  a 
CSG  with  twice  the  value  of  a  single  cruiser.  Therefore  the  presence  of  both  systems  should 
not  increase  the  acquisition  portfolio  value  for  these  capabilities  by  the  linear  sum  of  both 
systems’  utility.  Conversely,  the  anticipated  Air  and  Missile  Defense  Radar  (AMDR)  on  Flight 
III  Arleigh-Burke  destroyers  is  expected  to  provide  enhanced  value  beyond  the  linear 
aggregation  of  the  individual  ship  capacities  when  two  more  ships  operate  in  conjunction. 
While  MPT  does  not  consider  such  investment  interactions,  the  development  of  SoS  utility 
from  the  capabilities  of  constituent  systems  is  an  active  field  of  research.  Therefore,  this 
paper  adopts  two  possible  constructs  based  on  SoS  research  as  initial  attempts  to  address 
this  challenge  for  portfolio  design:  the  capability  tree  and  combination  coefficients. 

Introduction  of  the  Method  for  Portfolio  Affordability  Analysis 

Portfolio-Level  Epoch-Era  Analysis  for  Affordability  (PLEEAA)  is  proposed  as  a  new 
method  for  portfolio  affordability  analysis.  The  key  innovative  feature  of  the  method  is  the 
fusion  of  elements  from  Modern  Portfolio  Theory  with  the  capabilities  of  Epoch-Era  Analysis. 
Figure  2  displays  a  graphical  representation  of  how  this  is  achieved. 


Figure  2.  Portfolio-Level  Epoch-Era  Analysis  for  Affordability  (PLEEAA)  Process 

Model  for  Affordability  Analysis 

Multi-Attribute  Tradespace  Exploration  (MATE)  is  a  method  that  has  been  used 
extensively  in  system-level  design.  When  used  with  EEA,  MATE  may  design  for  a  variety  of 
“ilities,”  including  affordability.  As  shown  in  RSC,  MATE  combined  with  EEA  may  be  used  to 
consider  dynamic  uncertainty  of  contexts  and  needs.  PLEEAA  extends  RSC  with  two  new 
constructs  to  enumerate  viable  portfolios:  a  portfolio  selector  and  a  portfolio  design  tool. 
Additionally,  a  “portfolio  capability  tree”  is  also  implemented  to  link  system  performance  to 
portfolio  utility. 
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Portfolio  Design  Tool  and  Portfolio  Selector 

A  fundamental  element  of  MPT  is  “asset  allocation,”  or  the  identification  of  potential 
asset  classes  which  a  portfolio  designer  (the  systems  engineer  applying  PLEEAA)  desires  to 
consider  for  inclusion  in  the  portfolio.  In  finance,  these  assets  may  be  stocks,  bonds,  or  cash 
(Amenc  &  Sourd,  2005).  For  Navy  portfolios,  asset  allocation  may  involve  identifying 
fundamental  categories  of  assets  such  as  cruisers,  destroyers,  aircraft,  and  submarines,  for 
example.  The  portfolio  designer  may  apply  specific  constraints  to  each  class,  such  as  a 
maximum  number  of  units  (or  funds)  they  are  willing  to  allocate  to  that  class;  these  class 
constraints  mirror  investment  thresholds  in  financial  portfolios. 

The  asset  allocation  decisions  in  the  portfolio  design  tool  represent  the  most 
significant  lever  a  portfolio  designer  has  to  influence  the  outcome  of  the  PLEEAA  method.  If 
a  portfolio  designer  wishes  to  constrain  the  analysis  to  reflect  current  institutional  inertia  or 
design  paradigms,  they  may  develop  highly  specific  classes  and  class  constraints  which 
force  the  portfolio  selector  to  only  consider  designs  similar  to  existing  portfolios.  However,  if 
the  designer  wishes  to  explore  potentially  novel  approaches  and  unconsidered  emergent 
qualities  of  constituent  systems,  they  may  choose  to  set  few  to  no  class  constraints  on  the 
analysis.  This  would  enumerate  a  far  greater  portion  of  the  potential  portfolios  design  space, 
but  possibly  at  significant  computational  cost. 

A  second  role  of  the  portfolio  design  tool  and  class  constraints  is  to  reduce  the 
computational  complexity  of  the  portfolio  design  problem,  as  shown  in  the  case  example. 

Portfolio  Capability  Tree 

A  key  challenge  of  any  tradespace-based  portfolio  analysis  is  linking  constituent 
system  performance  attributes  with  portfolio-level  capabilities.  This  research  uses  the 
concept  of  a  capability  tree,  a  capability-based  value  mapping,  to  both  percolate  portfolio 
designer  needs  down  through  the  portfolio  levels  to  the  constituent  system  managers,  and 
then  to  amalgamate  system-level  performance  attributes  back  up  the  capability  tree  and 
define  a  portfolio-level  utility.  The  structure  and  inspiration  of  capability  trees  are  rooted  in 
means-ends  objectives  networks,  as  developed  by  Keeney  (1992).  Means-ends  objectives 
networks  link  decision  alternatives  to  their  impact  on  the  overall  objective  to  enable 
quantitative  modeling.  Capability  trees  seek  to  link  portfolio  asset  options  to  the  overall 
portfolio  utility  for  the  same  purpose. 

The  process  to  develop  the  capability  tree  begins  with  value-elicitation.  A  portfolio 
designer  decomposes  the  strategic  objectives  of  the  portfolio-level  stakeholders  into  a  set  of 
desired  capabilities,  or  performance  attributes.  Performance  attributes  become  a  measure  of 
how  well  a  potential  portfolio  meets  the  needs  of  the  portfolio  designer.  Each  performance 
attribute  of  the  portfolio  is  communicated  to  a  unique  program-level  manager  who  is  the 
decision  maker  for  constituent  programs  contributing  to  that  portfolio  performance  attribute. 
Similar  to  before,  the  program-level  manager  decomposes  their  values  into  a  set  of 
program-level  performance  attributes  which  are  communicated  to  either  another  program- 
level  manager,  or  to  the  constituent  systems  of  the  portfolio. 

It  is  important  to  recognize  that  the  portfolio  designer  and  program-level  managers  at 
each  level  in  the  portfolio  hierarchy  are  empowered  to  develop  their  own  performance 
attributes  and  mental  value  models  to  measure  the  operation  and  utility  of  the  systems  (or 
programs)  in  their  subtree.  The  utility  measurement  developed  is  then  shared  by  the  in-level 
manager  with  the  manager  one  level  above  in  the  capability  tree  hierarchy.  Therefore,  the 
utility  measurement  of  a  program  must  be  effectively  communicated  at  the  interface  of  the 
two  managers,  referred  to  as  a  “node.”  The  effective  communication  of  utility  between 
individuals  is  a  non-trivial  task,  and  in  general,  may  benefit  from  multi-party  utility 
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negotiations  such  as  those  under  research  by  Fitzgerald  and  Ross  (2015).  The  capability 
tree  model,  as  described  here,  is  an  imposed  constructed  value  model  and  is  illustrated  in 
Figure  3,  where  nodes  are  represented  by  collate  symbols. 

The  “nesting”  process  utilized  to  decompose  portfolio-level  strategic  objectives  into 
performance  attributes,  and  communicate  these  values  to  constituent  programs  or  systems, 
creates  a  two-dimensional  root  or  tree  structure  of  desired  performance  attributes  at  various 
levels  of  the  portfolio  hierarchy.  The  multi-document  symbol  in  Figure  3  represents  a 
program-level  manager  who  utilizes  their  own  value  and  performance  models  to  determine 
the  utility  of  the  constituent  systems  or  programs  in  their  subtree. 


|  to  of  tuwarchy 
|  to  ofhioardiy 

Each  program  or  portfolio- 
level  manager  has  their  own 
model  to  assess  utility  from 
the  constituen  t  program  or 
system  utilities 


•  •  • 


Each  “node"  entails  the 
handoff  of  information 
betw  een  program  or 
portfolio-level  managers 


Figure  3.  PLEEAA  “Capability  Tree”  Portfolio  Design  Architecture 

Through  the  capability  tree,  desired  portfolio  strategic  objectives  flow  down  to  lower 
level  program  managers  as  performance  attributes.  The  capability  trees  are  extendable  to 
multiple  levels  of  programs:  Three  have  been  shown  in  Figure  3.  Each  additional  level  of  the 
portfolio  hierarchy  allows  for  the  design  of  more  intricate  portfolios  through  TSE  techniques, 
but  may  also  substantially  increase  the  analysis  complexity.  All  “branches”  of  the  tree  do  not 
need  to  have  the  same  number  of  levels.  Some  capabilities  will  naturally  terminate  in 
system-level  attributes  after  the  first  level,  while  others  may  require  multiple  levels  of 
decomposition  to  equate  to  system-level  performance  attributes.  A  portfolio  objective  is  fully 
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decomposed  when  the  system-level  “leaf  node”  performance  attribute  is  readily  described 
by  a  metric  of  the  potential  component  systems,  such  as  the  number  of  missiles  on  a  ship. 

In  the  command  hierarchy  of  military  institutions,  utility  handoffs  may  be 
straightforward  due  to  the  subordinate  decision-making  (chain  of  command)  architecture. 

For  example,  the  structure  of  the  Joint  Capabilities  Integration  Development  System  (JCIDS) 
represents  the  information  hierarchy  that  the  capability  tree  seeks  to  leverage  for  portfolio 
affordability  analysis.  JCIDS  manages  the  DoD’s  joint  capability  portfolio  and  conducts 
capability  gap  assessments,  among  other  capability  requirement  development  and  approval 
functions,  by  identifying  joint  capability  areas  (similar  to  the  capability  tree  portfolio-level 
performance  capabilities)  and  engaging  decision  makers  in  a  four-tier  hierarchy  which 
moves  information  from  individual  subject  matter  experts  up  to  four  star  decision  makers 
(Chairman  of  the  Joint  Chiefs  of  Staff,  2015b).  In  a  very  real  sense,  the  PLEEAA  method 
adopts  the  portfolio  management  structure  of  the  JCIDS  process  established  by  Chairman 
of  the  Joint  Chiefs  of  Staff  (2015a)  and  integrates  MPT  and  EEA  tradespace  exploration 
techniques  to  produce  more  robust,  data-driven  portfolio  analyses. 

Complementary  and  Substitute  Systems 

A  fundamental  challenge  unique  to  engineering  portfolios  versus  financial  portfolios 
is  that  portfolio- 1  eve  I  capability  may  not  simply  be  an  aggregate  sum  of  the  constituent  asset 
capabilities.  As  discussed  previously,  while  a  portfolio  is  a  non-operational  construct  that 
describes  the  acquisition  and  inherent  values  of  a  set  of  assets,  engineering  portfolios  must 
consider  emergent  value  from  asset  interaction  in  a  SoS.  The  concept  of  complementary 
and  substitute  systems  in  SoS  has  been  an  area  of  intense  academic  focus.  To  provide  a 
few  definitions, 

1 .  Complementary  systems  are  two  or  more  constituent  systems  that 
experience  a  change  in  value  delivery  towards  existing  performance 
attributes,  or  gain  capability  in  new  performance  attributes,  when 
simultaneously  present  in  a  SoS  (in  this  case,  a  portfolio).  Such  changes  in 
value  usually  result  from  an  adjustment  in  the  CONOPS  of  the  systems.  The 
sign  and  magnitude  of  the  performance  change  is  variable.  For  example,  the 
missile  strike  capability  of  a  submarine  may  be  significantly  increased  when 
combined  with  the  advanced  radar  and  fire  control  capabilities  of  a  missile 
cruiser.  However,  the  same  submarine  may  experience  a  reduction  in  stealth 
capability  as  it  must  be  within  a  certain  range  of  the  cruiser  and  transmit 
targeting  information. 

2.  Substitute  systems  are  those  that  provide  an  overlapping  performance 
attribute  capability  in  an  operating  scenario  (i.e.,  CONOPS)  of  interest.  A 
guided  missile  cruiser  and  guided  missile  destroyer  may  perhaps  be 
considered  substitute  systems  in  terms  of  anti-ballistic  missile  capabilities. 
However,  they  would  not  likely  be  substitutes  for  littoral  operations  as  their 
capabilities  and  vulnerabilities  differ. 

The  PLEEAA  capability  tree  architecture  provides  a  unique  mechanism  to  identify 
and  assess  substitute  systems.  Since  all  potential  systems  are  simultaneously  evaluated  by 
each  bottom-level  portfolio  hierarchy  manager,  substitute  systems  that  provide  similar 
capability  are  likely  to  be  identified.  The  manager’s  value  model  may  then  appropriately 
determine  the  node’s  aggregate  utility  resulting  from  the  interacting  substitute  systems. 

The  identification  of  complementary  systems  is  not  as  straightforward,  but  is  still 
enabled  by  the  capability  tree  architecture.  Again,  because  bottom-level  portfolio  hierarchy 
managers  ideally  have  visibility  of  the  relevant  potential  constituent  systems  of  the  portfolio 
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that  contribute  to  the  performance  attribute  of  interest,  they  will  likely  be  able  to  identify 
system  pairings  which  complement  each  other’s  value  delivery.  The  bottom-level  managers, 
unlike  the  portfolio  designer  or  higher  level  program  managers,  will  reasonably  have  the 
expertise  to  understand  the  potential  interactions  between  systems.  In  other  words,  the 
capability  tree  framework  releases  portfolio-level  designers  from  trying  to  make  technical 
evaluations  on  the  numerous  constituent  systems  in  the  portfolio,  but  rather  allows  each 
manager  in  the  capability  tree  to  only  consider  system  or  program  interactions  and  values  at 
their  node.  If  a  bottom-level  portfolio  manager  is  unable  to  identify  system  interactions,  then 
emergent  complementary  value  may  not  be  properly  considered  in  the  portfolio  design. 

On  an  additional  note,  both  complementary  and  substitute  systems  are  likely  to 
represent  opportunities  for  cost  savings  through  joint  development  or  production  programs. 
While  this  information  is  unlikely  to  be  offered  by  the  constituent  system  operators  during 
value  elicitation,  the  bottom-level  managers  will  likely  recognize  such  potential.  Therefore, 
the  capability  tree  framework  may  allow  for  the  adjustment  of  both  utility  and  costs  with 
respect  to  complementary  and  substitute  system  interactions. 

While  the  capability  tree  framework  may  help  discern  the  influence  of  complementary 
and  substitute  systems,  the  approach  outlined  above  would  require  the  decision-making 
models  of  each  manager  to  be  well  represented  at  every  node.  For  this  initial  CSG  case 
study,  and  indeed  for  many  conceptual  design  or  acquisition  programs,  this  assumption  is 
not  realistic.  Therefore,  this  research  also  adopts  a  second  approach  to  manage 
complementary  and  substitute  systems  in  lower  fidelity  analyses. 

In  her  work  to  apply  tradespace  exploration  to  SoS,  Chattopadhyay  (2009) 
introduced  the  concept  of  “level  of  attribute  combination  complexity.”  Chattopadhyay 
identified  three  levels  of  combination  complexity  which  express  general,  first-order  estimates 
of  performance  attribute  interaction  from  different  constituent  systems  in  the  portfolio.  “Low- 
level  combination”  characterizes  performance  attributes  which  exist  independent  of  one 
another.  An  example  of  low-level  combination  may  be  an  E-2  Hawkeye  early  warning 
capability  and  a  Littoral  Combat  Ship  (LCS)  green-water  minesweeping  capability.  “Medium- 
level  combination”  describes  attributes  of  systems  which  contribute  to  the  delivery  of  the 
same  portfolio  performance  attribute,  but  are  characterized  by  a  sharing  of  value  deliver, 
such  as  through  a  handoff  of  responsibility.  The  handoff  of  targeting  information  from  a 
Ticonderoga  cruiser  to  a  Virginia-class  submarine  may  be  envisioned  as  medium-level 
combination  for  the  cruise  missile  strike  performance  attribute.  Finally,  “high-level 
combination”  are  system  attributes  which  interact  to  simultaneously  provide  a  portfolio 
performance  attribute,  such  as  the  anti-submarine  warfare  capabilities  of  a  Virginia-class 
submarine  and  an  Arleigh-Burke  destroyer.  A  given  system  may  contribute  to  performance 
attributes  at  one  or  all  of  these  levels  of  combination  complexity  when  considering  their 
interactions  with  other  systems. 

PLEEAA  elicits  bottom-level  portfolio  hierarchy  managers  to  characterize  the 
combination  complexity  for  each  performance  attribute  in  their  area  of  responsibility  for  each 
of  the  potential  constituent  systems.  Various  models  may  be  used  to  represent  the  impact  of 
the  level  of  combination  on  the  resulting  utility  and  cost  of  the  portfolio.  For  the  CSG  case 
study,  in  general,  low-level  combination  attributes  are  not  adjusted,  medium-level 
combination  attributes  utilize  alternative  models  for  utility  calculation,  and  high-level 
combination  attributes  receive  appropriate  utility  and  cost  multipliers. 

Overview  of  PLEEAA 

PLEEAA  is  a  variant  of  the  RSC  method  for  affordability  analysis  developed  by 
Schaffner  et  al.  (2013).  The  Gather-Evaluate-Analyze  structure  of  the  RSC  method  was 
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maintained  and  supplemented  with  additional  steps  necessary  for  portfolio  design. 
Specifically,  the  value-driven  design  formulation  now  occurs  with  multiple  groups  of 
stakeholders  corresponding  to  different  levels  of  the  portfolio  hierarchy. 

The  basic  steps  of  PLEEAA  are  represented  as  an  1 1 -process  method  shown  in 
Figure  4.  Processes  1  through  5  involve  value  elicitation  of  multiple-levels  of  stakeholders  in 
the  portfolio  to  define  the  problem  scope,  assess  stakeholder  needs,  identify  contextual 
variables,  and  assess  combination  complexity  information  for  potential  systems.  Process  6 
conducts  the  composite  EEA/MPT  analysis  to  produce  a  tradespace  of  potential  portfolio 
designs  for  the  considered  epoch.  Before  proceeding  to  the  following  processes,  feedback  is 
provided  to  the  designers  and  stakeholders  to  allow  for  adjustments  in  the  provided 
information,  as  necessary.  This  feedback  loop  is  a  key  element  of  TSE  as  stakeholder 
values  may  change  as  portfolio  options  and  tradeoffs  are  made  clear.  Finally,  Processes  7 
through  1 1  support  designers  at  the  portfolio-level  to  compare  the  dynamic  properties  of 
potential  portfolios  in  light  of  their  anticipated  performance  in  a  variety  of  point  futures 
(epochs),  as  well  as  possible  lifecycle  narratives  (eras).  These  processes  are  described  and 
applied  to  the  CSG  study  in  the  following  section. 


Information  Gathering 
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Context  Definition 

2.  Portfolio-level 
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Figure  4.  A  Graphical  Overview  of  the  Modified  Gather-Evaluate-Analyze 

Structure  for  PLEEAA 

Demonstration  Case:  Carrier  Strike  Group  Design  for  Affordability 

According  to  a  2006  RAND  study,  the  cost  growth  rates  for  new  naval  units  such  as 
“amphibious  ships,  surface  combatants,  attack  submarines,  and  nuclear  aircraft  carriers 
have  ranged  from  7  to  1 1  percent,”  an  inflationary  rate  which  significantly  outstrips 
development  costs  in  other  sectors  (Arena  et  al.,  2006).  In  the  decade  since  this  report,  the 
severe  cost  overruns  in  the  Littoral  Combat  Ship  (LCS)  and  Zumwalt-class  destroyer 
(DDG1000)  programs  have  likely  exacerbated  this  figure.  In  light  of  this  matter,  it  appears 
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appropriate  to  apply  PLEEAA  to  an  application  of  large  ship  acquisitions  in  order  to  display 
the  potential  of  this  method  to  assist  decision  problems  in  the  following  ways: 

•  As  the  LCS  and  DDG1000  programs  are  scaled  back  (or  eliminated)  and 
replaced  with  alternative  systems,  portfolio-level  affordability  analysis  may  aid 
the  identification  of  resulting  capability  gaps  in  the  Navy’s  strategic  portfolio. 

•  The  use  of  a  PLEEAA  may  facilitate  the  definition  of  acquisition  program 
requirements  which  limit  capability  creep,  such  as  what  was  seen  in  the  LCS 
development. 

•  Portfolio-level  affordability  analysis  will  give  decision  makers  insight  into  the 
value  tradeoff  of  investing  in  high-capability,  high-costs  systems,  versus  low- 
capability,  low-cost  systems  for  future  naval  group  operations. 

•  EEA,  when  applied  to  a  ship  acquisition  portfolio,  will  allow  designers  to 
foresee  the  affordability  of  the  proposed  capability  portfolio  in  multiple 
potential  future  scenarios. 

Considering  these  potential  benefits  to  the  naval  acquisition  process,  the  design  of  a 
portfolio  of  systems,  from  which  a  CSG(s)  may  be  developed,  presents  itself  as  an 
appropriate  case  study  to  demonstrate  PLEEAA.  Beyond  the  relevance  of  the  analysis  to 
the  DoD’s  goals  of  an  affordable  Navy  and  the  emergence  of  new  tactics  and  asymmetric 
threats  for  exiting  CSGs,  the  complexity  the  CSG  design  problem  is  conveniently  reduced 
through  a  series  of  realistic  assumptions.  The  simplifying  assumptions  adopted  include  the 
following: 

1 .  A  CSG  is  a  directed  SoS  as  defined  by  Maier  (1999),  in  which  a  central 
authority  (the  combatant  commander  [CCDR]  and  operational  commander) 
have  decision  authority  over  the  constitute  systems  (Chief  of  Naval 
Operations,  2010).  A  directed  SoS  is  ideal  for  the  initial  application  of  this 
method  as  the  additional  complexities  inherent  to  incomplete  managerial 
influence  or  decentralized  control  are  avoided  (Shah,  2013). 

2.  The  baseline  composition  and  basic  mission  capabilities  of  CSGs  are  well- 
defined  by  Chief  of  Naval  Operations  (2010),  enabling  effective  bounding  of 
the  potential  portfolio  asset  set  for  the  purposes  of  this  case  study. 
Furthermore,  a  CSG  is  intended  to  function  autonomously  for  many  of  its 
operations.  This  further  simplifies  the  scoping  of  the  portfolio  boundary 
conditions  and  interfaces. 

3.  The  hierarchy  of  the  Navy  designates  specific  decision-making  authority  to 
specific  individuals.  This  structure,  including  the  Navy’s  use  of  subject  matter 
experts,  is  directly  paralleled  by  the  capability  tree  structure  of  PLEEAA. 
Therefore,  in  the  analysis,  each  node  of  the  capability  tree  may  be  mapped  to 
a  specific  decision  maker  in  the  CSG  command  structure. 

The  following  sections  briefly  describe  each  process  in  PLEEAA.  A  representational 
outcome  is  included,  which  provides  a  first  pass,  high-level  application  of  PLEEAA  to  CSG 
portfolio  design,  subject  to  the  assumptions  highlighted  previously  in  this  section.  The  values 
of  stakeholders,  models  for  performance  aggregation,  and  system  performance  parameters 
were  notionally  created  from  publically  available  information  and  feedback  from  experts 
familiar  with  the  systems. 

Process  1:  Value-Driving  Context  Definition 

The  first  process  in  PLEEAA  begins  by  identifying  the  basic  problem  statement  and 
design  space  for  the  proposed  portfolio.  The  portfolio-level  stakeholders  are  identified  and 
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engaged  as  necessary  to  formulate  relevant  exogenous  uncertainties  and  outline  initial 
value  propositions.  An  inchoate  set  of  potential  constituent  systems  is  constructed,  and  the 
portfolio  designer’s  degree  of  influence  over  these  systems  is  predicted. 

Representational  Outcome :  According  to  the  Chief  of  Naval  Operations  (2010),  the 
primary  portfolio-level  stakeholders  influencing  the  design  of  a  CSG  are  the  combatant 
commander  (CCDR)  and  operational  commander  of  the  naval  group.  The  CSG  value 
proposition  is  outlined  therein  as  a  “responsive,  flexible  capability  for  sustained  maritime 
power  projection  and  combat  survivability  to  shape  the  operation  environment,  respond  to 
crises,  and  protect  the  United  States  and  allied  interest  in  any  threat  environment.”  An  initial 
set  of  12  potential  constituent  systems  is  provided  in  the  work  instruction.  Seven  more 
systems  were  added  to  capture  the  NGCS  work  conducted  in  prior  research  and  the  new 
Zumwalt-class  destroyer.  For  this  analysis,  the  portfolio  designer  is  assumed  to  have  total 
control  over  the  acquisition  of  new  systems  for  the  CSG  portfolio  (subject  to  the  defined 
constraints). 

Process  2:  Portfolio-Level  Stakeholder  Value-Driven  Design  Formulation 

In  Process  2,  the  portfolio  designer  elicits  a  variety  of  information  from  the  portfolio- 
level  stakeholders  to  establish  the  root  (top  level)  of  the  capability  tree  and  clarify  the 
constraints  of  the  portfolio  design  and  composition. 

•  Performance  Attributes:  A  set  of  overarching  capabilities  which  the  portfolio 
must  be  able  to  fulfil  to  meet  the  strategic  objectives.  These  capabilities,  or 
performance  attributes,  are  assigned  weights  based  on  elicited  information  to 
reflect  the  stakeholder  preferences.  A  utility  function  is  developed. 

•  Expense  Attributes:  Portfolio  resource  statements  are  translated  into  specific 
costs  measured  in  the  portfolio.  Acceptable  expense  thresholds  are  identified, 
and  expense  functions  are  created. 

•  Portfolio  Investment  Strategy  Constraints:  Constraints  are  set  concerning 
viable  portfolio  composition.  These  constraints  may  define  limitations  on 
resources,  types  of  constituent  systems,  and  acceptable  risk.  They  could  also 
be  more  specific  and  govern  internal  investment  strategy  decisions  such  as 
the  maximum  number  of  component  systems  allowed  or  the  minimum 
resource  allocation  value  to  any  single  system. 

Representational  Outcome :  Simplified  interpretations  of  notional,  portfolio-level 
performance  and  expense  attributes  for  a  CSG  portfolio  are  shown  in  Table  3.  For  this  initial 
case  study,  the  performance  and  expense  attributes  yield  multi-attribute  utility  and  multi¬ 
attribute  expense  through  linear  weighted  sum  aggregation  functions.  The  value  weights  at 
each  node  sum  to  one.  This  requires  an  assumption  that  the  performance  and  expense 
attributes  contribute  independently  to  aggregate  value.  While  this  is  not  necessarily  a 
realistic  assumption  for  a  CSG  portfolio,  it  is  sufficient  first  order  estimation  for  the 
demonstration  purposes  of  this  case  study. 
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Table  3.  Portfolio-Level  Performance  and  Expense  Attributes  for  a  CSG 


Performance  Att 

Notional  portfolio  strategic  objective 

Electronic  Warfare 

Cap  ability 

Suppress  enemy  EM  spectrum  capabilities  to  detect  friendly  assets  while  providing 
constant  functionality  despite  enemy  EM  countermeasures 

Defensive  Capability 

Maintain  asset  functionality  against  enemy  attack  and  countermeasures 

Offensive  Capability 

Provide  offensive  capabilities  both  within  and  beyond  the  battlespaee 

Power  Projection 

Project  psychological  power  in  the  sphere  of  influence  of  the  CSG  to  support 
peacekeeping  missions,  deter  attack,  and  increase  the  prestige  of  the  U  S. 

Logistics 

Replenish  the  consumables  of  the  CSG  anywhere  in  the  world  at  appropriate  intervals 
in  combat  and  non-combat  situations.  The  CSG  shall  be  capable  of  operation  for  a 
reasonable  period  of  time  without  re-supply 

Expense  Attribute 

Notional  portfolio  strategic  objective 

Acquisition  Cost 

The  cost  of  producing  new  constituent  system 

influence  Cost 

The  cost  incurred  to  leverage  a  constituent  system  to  participate  in  the  CSG 

Operation  Cost 

The  annual  cost  of  a  constituent  system 

Schedule  Cost 

The  perceived  cost  to  portfolio  designers  of  constituent  system  late  entry  into  the 
portfolio 

Process  3:  Capability  Tree  Stakeholder  Value-Driven  Design  Formulation 

From  the  information  gathered  in  the  previous  processes,  the  portfolio  designer 
creates  a  notional  structure  of  the  capability  tree  by  identifying  specific  program  managers  at 
each  node.  As  a  reminder,  the  manager  at  each  node  controls,  or  has  expertise  in,  a 
performance  capability  identified  in  the  higher  level  of  the  portfolio.  The  portfolio  designer 
elicits  a  variety  of  information  from  each  program  manager  of  the  capability  tree  and 
continues  the  process  until  all  branches  have  terminated  into  the  system-level  attributes,  or 
leaf  nodes. 

Representational  Outcome :  The  capability  tree  for  the  CSG  case  study  attests  to  the 
inherent  complexity  of  portfolio  analysis.  For  while  the  representation  of  the  CSG  has  been 
simplified,  the  corresponding  capability  tree  developed  contains  five  branches 
(corresponding  to  the  five  portfolio-level  performance  attributes  from  Process  2)  and, 
through  up  to  four  levels  of  decomposition,  91  distinct  system-level  performance  attributes. 
For  the  sake  of  brevity,  the  entire  capability  tree  has  not  been  included  in  this  paper; 
however,  Figure  5  displays  the  tree  with  all  but  the  system-level  performance  attributes 
included.  It  should  be  noted  that  some  branches  terminate  after  only  the  portfolio-level 
attributes,  while  other  branches  decompose  capabilities  through  three  nodes  before 
reaching  system-level  attributes. 
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Portfolio  Level  Capability  Attributes 
2nd  Level  SoS  Capability  Attributes _ 

3rd  Level  SoS  Capability  Attributes _ 

tt  of  System  Level  Performance  Attributes 


Levels  of  capability  tree 
hierarchy 


CSG  Electronic  Warfare  Capability 

Early  Warning 

6 

Weapon  system  detection 

6 

Electromagnetic  System 

5 

CSG  Defensive 
Capability 

Battlespace  Defense 
Capability 

Sea  superiority 

5 

Air  Superiority 

5 

Undersea  Superiority 

5 

Combat  Search  and  Rescue 

2 

Naval  Asset  Defense 
Capability 

Anti-Ship  Missile  Defense 

5 

Anti-Ship  Kinetic  Weapon 

5 

Sea  Mine  Defense 

5 

Torpedo  Defense 

5 

Crew  Defense 

4 

Naval  Gun  Fire  Support 

5 

CSG  Offensive 

Missile  Strike 
Capability 

Ballistic  Missile  interception 

5 

Cruise  Missile  Strike 

5 

Capability 

Torpedo  Capability 

5 

Sea  Basing  Capability 

3 

Special  Forces  insertion 

4 

Power  Projection  2 

CSG  Logistics  4 

Branch  of 
capability 
tree 


Note.  Read  tree  from  left  to  right. 


Figure  5.  CSG  Case  Study  Capability  Tree  Outline 
Process  4:  Epoch  Characterization 

All  stakeholders  related  to  the  portfolio  (portfolio  decision  makers,  program 
managers,  system  operators,  external  stakeholders,  etc.)  are  engaged  to  identify  possible 
key  future  contexts  that  might  impact  success,  and  to  characterize  the  uncertainty  of  each 
context.  It  is  anticipated  that  a  core  set  of  contextual  uncertainties  will  emerge  from  the 
stakeholders  that  may  be  parameterized  by  a  common  set  of  epoch  variables.  Any 
anticipated  changes  in  stakeholder  preferences  (performance  attribute  weightings)  are 
identified. 


Representational  Outcome:  Seven  epoch  variables  were  identified  from  five  major 
categories  of  uncertainty  (technology  levels,  maintenance  events,  policy  environment,  SoS 
management  abilities,  and  CSG  threats).  Table  4  displays  the  range  of  uncertainty 
represented  by  each  epoch  variable  as  well  as  units  of  measurement.  The  technology  epoch 
variables  most  directly  influence  constituent  system  performance.  The  maintenance,  policy 
and  SoS  management  epoch  variables  most  directly  influence  the  cost  functions.  The 
threats  epoch  variables  change  the  stakeholder  preferences  for  portfolio  attribute 
performance. 
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Table  4.  Contextual  Uncertainties  of  CSG  Captured  in  Epoch  Variables 


EV  Category 

Epoch  Variable 

[Range] 

Units 

EV- Technology 

Advanced  Energy  Weapons 
(AEW) 

[0,  5,  50] 

MW 

EV- Technology 

Unmanned  Aerial  Systems  (UAS) 

[2,  10.  50] 

Berths 

EV  -  Maintenance 

Overhaul  Event  Costs 

[0,  0.5e9,  2e9] 

Billions  $ 

EV  -  Policy 

Budget 

[80,  100,  150] 

% 

EV-SoS 

management 

Cooperation  Costs 

[80,  100,  150] 

% 

EV  -  Threats 

Enemy  Threat 

[Low,  Med,  High] 

Level 

EV  -  Threats 

Asymmetric  Threat 

[Low,  Med,  High] 

Level 

For  the  initial  case  study  presented  in  this  paper,  the  technological  epoch  variables 
were  excluded  to  simplify  the  analysis  and  focus  the  results  on  affordability  considerations  of 
the  variance  in  portfolio  cost  attributes. 

The  five  remaining  epoch  variables  were  enumerated  independently  and  combined 
to  produce  a  total  of  243  potential  epochs.  For  the  sake  of  simplicity,  five  possible  epochs 
that  represent  recognizable  potential  futures  were  extracted  from  this  set  in  a  “narrative” 
sampling  approach.  These  five  epochs  and  their  epoch  variable  levels  are  provided  in  Table 
5. 


Table  5.  Epoch  Construction  From  Epoch  Variables  for  Five  Selected  Epochs 


Epoch  Names 

Overhaul 

Budget 

Epoch  Variables 

Cooperation  Enemy 

Asymmetric 

Baseline  (BL) 

0 

100 

100 

Low 

Med 

Small  Navy  (SN) 

0 

80 

150 

Low 

Low 

War  on  Terror 
(WoT) 

0 

100 

80 

Low 

High 

Major  Conflict  (MC) 

0 

150 

80 

High 

Med 

Peacekeeping  (PK) 

0.5e9 

100 

100 

Med 

Med 

Process  5:  System-Level  Capability  Assessment 

Each  potential  constituent  system  must  be  evaluated  for  its  performance  in  each 
system-level  performance  attribute.  This  “capability  assessment”  may  take  a  variety  of 
forms.  In  some  cases,  the  bottom-level  capability  tree  hierarchy  managers  may  utilize 
performance  models  to  computationally  assess  the  potential  constituent  systems’ 
performance.  In  other  cases,  however,  such  models  may  not  exist  and  the  bottom-level 
managers,  or  an  appropriate  subject  matter  expert,  shall  be  engaged  to  assess  system 
performance  qualitatively.  Each  constituent  system  must  be  assessed  for  all  performance 
attributes  in  each  epoch. 

Representational  Outcome :  The  notional  performance  of  the  1 9  potential  constituent 
systems  was  assessed  for  each  of  the  91  system-level  performance  attributes  in  the  five 
potential  epochs.  Publically  available  information  was  utilized  to  assign  performance  on  a  0, 

1 ,  3,  9  scale;  0  represented  no  performance,  1  was  minimal  performance,  3  was  moderate 
performance,  and  9  was  performance  sufficient  to  meet  the  desired  portfolio  capability.  The 
CSG  capability  assessment  results  were  reviewed  for  reasonableness  by  individuals  familiar 
with  naval  systems. 
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Process  6;  Design-Epoch-Era  Tradespace  Evaluation 

A  key  high-level  summary  visualization  is  a  tradespace  with  axes  of  Multi-Attribute 
Utility  (MAU)  versus  Multi-Attribute  Expense  (MAE),  as  demonstrated  by  Wu  (2014). 
However,  unlike  in  traditional  TSE,  the  points  inside  a  portfolio  tradespace  do  not  represent 
single  systems  or  programs,  but  rather  represent  unique  combinations  of  assets  determined 
by  the  portfolio  investment  strategy.  The  process  flow  of  PLEEAA  depicted  in  Figure  2 
highlights  the  process  utilized  to  design  portfolios,  find  portfolio  MAU  and  MAE,  evaluate  the 
validity  of  the  result,  and  create  the  data  necessary  for  depicting  a  tradespace  of  viable 
alternatives. 

Representational  Outcome :  Without  the  ability  to  elicit  performance  attribute 
aggregation  models  from  each  of  the  program-level  managers,  this  research  developed  a 
series  of  six  functions  to  evaluate  bottom-level  portfolio  manager  utility  and  cost  attributes 
from  the  constituent  systems  in  a  portfolio.  Each  performance  and  cost  attribute  of  the 
portfolio  was  assigned  one  of  the  six  functions  depending  upon  which  one  best  represented 
the  notional  program-level  manager  value.  Once  the  bottom-level  performance  and  cost 
attributes  had  been  calculated,  the  value  was  multiplied  by  the  preference  weighing  of  the 
program-level  manager  at  the  node  in  the  next  level  of  the  portfolio  hierarchy.  This  process 
was  repeated  for  each  level  of  the  portfolio  hierarchy  until  all  branches  were  aggregated  to 
the  portfolio-designer.  The  portfolio-level  preference  weightings  were  then  applied  to  find  the 
ultimate  utility  and  costs  of  the  portfolio.  Table  6  provides  information  on  the  approach  and 
result  for  utility  evaluation  at  each  level  of  the  capability  tree. 

Table  6.  Approach  to  Derive  Portfolio-Level  Utility  From  System  Performance 

Attributes  Through  the  Capability  Tree 


Portfolio  Designer 

2nd  Level  Program 
Manger 

3rd  Level  Program 
Manager 

System  Performance 
Attributes 

Approach 

Applied  linear-weight 
sum  aggregation  to 
the  utility  from  each 

2^  level  program 
manager 

Applied  linear-weight 
sum  aggregation  to  the 
utility  from  each  3rd 
level  program  manager 

Applied  one  of  six 
models  which  best 
assess  3rd  level  program 
manager  utility  from  the 
constituent  systems 

Each  system  was 
evaluated  on  0,1. 3,9 
scale  for  each 
performance  attribute 

Result 

Portfolio  utility  on  a  0 
to  1  scale 

2nd  level  program  utility 
on  a  0  to  1  scale 

3rd  level  program  utility 
on  a  0  to  1  scale 

System  performance 
attribute  values 

The  PLEEAA  method  enumerated  53,018,336  possible  portfolios  and  identified 
524,160  portfolios  which  met  the  class  constraints.  This  subset  of  portfolios  was  then 
evaluated  according  to  the  stakeholder  preferences  and  constituent  system  performance  of 
each  epoch.  Table  7  displays  the  number  of  valid  portfolios,  or  portfolios  which  met  the  MAU 
and  MAE  constraints,  and  the  percentage  of  the  total  potential  designs  that  were  found  to  be 
feasible.  The  Small  Navy  epoch  is  the  most  limiting  epoch  due  to  its  80%  budget  and  150% 
cooperation  costs. 

Table  7.  Single-Epoch  Tradespace  Evaluation  Summary 


Epoch 

Valid  Portfolios 

Yield 

Baseline 

156,750 

29.9% 

Small  Navy 

420 

0.08% 

War  on  Terror 

156,750 

29.9% 

Major  Conflict 

492,400 

93.9% 

Peacekeeping 

156,750 

29.9% 

MSi 


-219- 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


Process  7:  Single  Epoch  Analysis 

The  valid  portfolios  for  each  epoch  may  be  plotted  in  a  tradespace  of  MAU  versus 
MAE.  By  exploring  each  single-epoch  tradespace,  a  designer  can  identify  which  portfolio 
design  options  for  the  CSG  perform  particularly  well  for  the  context  represented  in  that 
epoch,  and  what  constituent  systems  Pareto  efficient  portfolios  have  in  common. 

Representational  Outcome :  The  tradespace  of  viable  CSG  portfolios  for  the  Baseline 
epoch  is  provided  in  Figure  6.  The  Pareto  frontier  of  the  Baseline  epoch  contains  a  total  of 
26  potential  portfolio  designs.  The  specifications  for  five  example  Pareto  optimal  portfolios 
have  been  provided.  Figure  7  visually  displays  the  composition  of  the  promising  portfolios 
and  highlights  constituent  system  investments.  The  constituent  system  types  which  differ 
between  the  promising  portfolios  were  identified  with  Portfolio  A  as  a  reference.  This 
portfolio  comparison  is  intended  to  reveal  to  what  degree  the  same  constituent  systems 
appear  in  various  promising  portfolios.  Two  portfolios  which  have  the  same  types  of 
constituent  system,  such  as  Portfolios  A  and  C,  are  identified  as  having  no  different  systems 
despite  possessing  varying  numbers  of  each  system  type. 


# 

MAL 

MAE 

Portfolio  Composition 

A 

0.6536 

00352 

!  Arleigh  Burke  Flight  III 

1  Nimitz  with  Complemmt 

I  Los  Angles 

B 

0.7223 

0.14S7 

I  Arleigh  Burke  Flight  £ 

I  Arleigh  Burke  Flight  II 

I  Arleigh  Buikc  Flight  III 

1  Nimitz  with  Complement 
l  Los  Angles 

C 

0.754S 

0.1955 

2  Arleigh  Burk  e  Flight  111 

I  Nimitz  with  C  o  mpl  email 

I  Los  Angles 

D 

0.7942 

0,3072 

2  Arleigh  Buike  Flight  I 

2  Arleigh  Burke  Flight  III 

I  Nimitz  with  C  omplement 

I  Los  Angles 

E 

O.S35S 

0.6660 

2  NGCS  Variant  6 

2  Arleigh  B  Flight  II  Upgrade 

2  Arleigh  Buike  Flight  III 

1  Nimitz  with  Complement 

I  Los  Andes 

Figure  6.  Portfolio  Affordability  Tradespace  (MAU  v.  MAE)  and  Pareto  Efficient 

Portfolios  for  the  Baseline  Epoch 


Figure  7.  Constituent  System  Comparison  for  Promising  Portfolios  With  Portfolio 

A  as  the  Reference  Portfolio 


MSi 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-220- 


Process  8:  Multi-Epoch  Analysis 

A  fundamental  technique  within  EEA  is  the  comparison  of  potential  portfolios  across 
multiple  epochs.  Following  single-epoch  portfolio  analysis,  a  general  understanding  of 
performance  of  the  SoS  in  each  epoch  has  been  established.  By  comparing  the 
performance  of  each  portfolio  across  multiple  epochs,  various  metrics  can  be  utilized  to 
assess  the  design’s  robustness  against  change  and  uncertainty,  or  how  well  a  single 
portfolio  can  maintain  its  value  across  multiple  epochs.  The  reader  should  consult  Schaffner 
et  al.  (2013)  for  a  detailed  explanation  of  multi-epoch  analysis. 

Representational  Outcome :  A  particularly  useful  concept  to  assess  the  performance 
of  a  promising  portfolio  design  across  multiple  epochs,  or  to  discover  passive  robust 
solutions,  is  the  concept  of  Normalized  Pareto  Trace  (NPT;  Ross  et  al.,  2009).  An  NPT 
score  is  assigned  to  each  portfolio  design  of  interest  and  describes  the  percentage  of 
epochs  in  which  that  design  constitutes  a  Pareto  optimal  point.  A  variant  of  NPT,  the  fuzzy 
NPT  (fNPT),  reveals  the  percentage  of  epochs  for  which  a  particular  design  is  within  a 
certain  threshold  factor  of  the  Pareto  front.  The  width  of  this  threshold  zone  is  defined  by  a  K 
factor,  where  a  K  factor  of  zero  indicates  0%  fuzziness  and  is  identical  to  the  NPT  metric. 
Figure  8  illustrates  the  concept  of  fNPT,  and  Table  8  displays  the  NPT  and  fNPT  measures 
for  the  five  portfolios  identified  in  Figure  6. 


Figure  8.  Illustration  of  the  fNPT  Metric,  Where  K  Is  the  Threshold  Width  Applied 

Below  the  Pareto  Frontier 


Table  8.  The  NPT  and  fNPT  Metrics  for  Five  Promising  CSG  Portfolio  Designs 

Over  the  Five  Representative  Epochs 


Portfolio 

NPT 

5%  fNPT 

10%  fNPT 

20%  fNPT 

A 

1 

1 

1 

1 

B 

0.8 

0.8 

0.8 

0.8 

C 

0.6 

0.8 

1 

1 

D 

0.8 

0.8 

08 

08 

E 

0.4 

0.8 

0.8 

0,8 

Table  8  displays  that  Portfolio  A  has  an  NPT  value  of  one.  This  indicates  that 
Portfolio  A  remains  on  the  Pareto  frontier  in  all  epochs  considered.  From  Figure  6,  it  is 
apparent  that  Portfolio  A  also  represents  the  lowest  expense  and  utility  of  any  of  the 
promising  portfolio  designs.  Meanwhile  the  NPT  for  Portfolio  C  is  0.6,  indicating  that  in  60% 
of  the  epochs,  the  design  is  Pareto  optimal.  However,  Portfolio  C  reaches  an  fNPT  value  of 
one  at  a  K  value  of  10%.  This  means  that  Portfolio  C  is  within  10%  of  the  Pareto  frontier  in 
terms  of  MAU  and  MAE  for  all  epochs.  These  two  portfolios  (A  and  C)  are  the  most 
passively  robust  to  the  uncertainties  in  the  considered  epochs.  No  other  considered  portfolio 
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beyond  A  or  C  has  an  fNPT  value  of  one,  even  at  a  K  value  of  20%.  This  indicates  that  the 
other  promising  portfolios  are  significantly  removed  from  the  Pareto  front  in  at  least  one 
epoch. 

Process  9:  Era  Construction 

Another  useful  technique  in  EEA  involves  the  concept  of  evaluating  portfolio  designs 
over  an  ordered  set  of  epochs,  called  an  era,  to  represent  a  potential  lifecycle  of  the 
portfolio.  Era  analysis  enables  the  designer  to  understand  how  portfolio  designs  could 
maintain  their  value  through  the  uncertainty  of  a  long  run  potential  future.  Disturbances  and 
degradation  from  an  earlier  epoch  in  the  era  may  diminish  the  long  term  value  of  a  particular 
portfolio  design  with  respect  to  another.  Time-dependent  concerns,  such  as  cumulative 
carrying  costs  and  time  to  initial  operating  capabilities  of  various  assets,  can  be  considered 
during  era  analysis.  While  such  time-dependent  factors  were  not  considered  as  part  of  the 
initial  CSG  case  study,  they  are  an  anticipated  area  of  future  research. 

Representational  Outcome'.  Eras  may  be  developed  through  stakeholder  elicitation, 
probabilistic  modeling  (e.g.,  Monte  Carlo  or  Markov  models),  or  through  a  narrative 
approach  to  produce  likely  potential  futures.  Two  eras  were  created  for  this  case  study.  A 
narrative  approach  was  used  to  select  the  order  and  duration  of  the  epochs  to  represent  a 
potential  30-year  operational  life  for  the  carrier  strike  group: 

Era  1:  Baseline  (5  yr),  War  on  Terror  (5  yr),  Peacekeeping  (10  yr),  Baseline  (3  yr), 
Small  Navy  (7  yr) 

Era  2:  Peacekeeping  (5  yr),  Major  Conflict  (7  yr),  Peacekeeping  (10  yr),  Small  Navy 
(5  yr),  Baseline  (3  yr) 

Process  10:  Single-Era  Analysis 

In  single  era  analysis,  the  performance  of  portfolio  designs  is  assessed  over  a 
sequence  of  epochs  as  described  in  Process  9.  Single-era  analysis  enables  designers  to 
understand  the  time-ordered  effects  of  the  epochs  on  the  portfolios.  This  allows  for  the 
identification  of  portfolios  which  maintain  utility  and  affordability  throughout  the  sequence  of 
potential  futures,  and  of  those  which  may  become  unaffordable.  The  concepts  of  NPT  and 
fNPT  may  also  be  applied  to  an  era  to  quantitatively  measure  how  well  a  particular  portfolio 
design  compares  to  the  Pareto  front  solutions  of  each  epoch  in  the  era. 

Representational  Outcome :  For  the  CSG  portfolio  design  study,  both  the  variance  of 
utility  and  expense  of  a  potential  portfolio  over  the  lifecycle  of  the  CSG  are  of  interest  to  a 
potential  designer.  Therefore,  Figure  9  contains  two  subplots  which  display  the  change  in 
MAU  and  MAE  over  the  five  epochs  for  the  five  Pareto  efficient  portfolios  in  the  Baseline 
epoch.  Tracing  the  trajectories  reveals  the  emergent  affordability,  or  unaffordability,  of  the 
potential  CSG  portfolios. 
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Figure  9.  (a)  Expense  Considerations  for  Era  1;  (b)  Utility  Considerations  for  Era  1 

The  purpose  of  this  initial  case  study  was  to  explore  the  ability  of  PLEEAA  to  support 
design  for  affordability.  The  case  study  was  therefore  constructed  to  focus  on  exogenous 
factors  anticipated  to  impact  expense  attributes.  As  a  result,  the  MAE  of  the  portfolios 
exhibits  significant  variance  over  the  era  in  Figure  9(a).  However,  there  is  relatively  little 
variation  in  the  MAU  values  between  the  selected  portfolios  through  all  epochs  in  Figure 
9(b).  The  utility  variation  is  small  because  the  technology  epoch  variables,  which  most 
significantly  influence  constituent  system  performance,  were  not  included  in  this  initial  study. 
Additionally,  the  class  constraints  of  this  analysis  required  all  portfolios  to  have  an  aircraft 
carrier,  a  submarine,  and  at  least  one  multi-mission  capable  surface  combatant.  The  class 
constraints  therefore  provided  a  relatively  high  minimum  utility  of  all  valid  portfolios  which 
further  reduces  apparent  variation.  Finally,  as  illustrated  in  Figure  7,  the  promising  portfolios 
primarily  contained  multiple  units  of  the  same  constituent  systems.  As  a  result,  the  portfolios 
exhibit  similar  utility  responses  to  the  uncertainties  modeled.  Future  research  will  include 
epoch  variables  which  strongly  impact  capability  performance  attributes  in  order  to  more 
fully  represent  the  design  problem. 

Process  11:  Multi-Era  Analysis1 

Multi-era  portfolio  analysis  expands  single-era  analysis  by  identifying  patterns  of 
affordability  (and  unaffordability)  as  well  as  utility  delivery  that  emerge  from  the  path 
dependent  development  of  portfolios  through  multiple  contexts.  This  process  enables  a 
designer  to  develop  metrics  that  characterize  the  affordability  of  potential  portfolio  designs 
across  a  variety  of  potential  lifecycles.  The  reader  should  consult  Schaffner  et  al.  (2013)  for 
a  detailed  explanation  of  multi-era  analysis. 

Discussion  of  the  “Affordable”  CSG  Design  Through  PLEEAA 

There  are  hundreds,  if  not  thousands,  of  factors  that  influence  the  lifecycle  design  of 
a  CSG  portfolio.  While  it  may  not  be  possible  to  ever  consider  each  of  them,  modern-day 
computing  capabilities  and  systems  engineering  methodologies  enable  a  designer  to 
consider  far  more  of  these  influence  factors  during  conceptual  design  than  was  traditionally 


1  This  process  was  not  demonstrated  by  the  current  study  due  to  the  representative  nature  of  the 
analysis,  but  is  described  here  for  completeness. 
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possible.  PLEEAA  provided  a  method  to  consider  91  distinct  CSG  system-level  performance 
attributes  and  relate  their  value  back  up  to  the  overall  portfolio  utility.  The  performance 
attributes  and  expense  attributes,  along  with  the  preferences  of  the  relevant  stakeholders, 
were  defined  in  PLEEAA  for  five  epochs  to  characterize  contextual  uncertainty  of  potential 
future  states. 

PLEEAA  supported  the  identification  of  CSG  designs  which  appear  on  the  Pareto 
frontier  of  each  epoch  through  single-epoch  analyses.  Furthermore,  utilizing  the  metrics  of 
NPT  and  fNPT  in  multi-epoch  analysis,  PLEEAA  identified  five  promising  portfolios  which 
exhibit  acceptable  performance  and  affordability  across  a  majority  of  the  epochs  considered. 
From  Table  8,  it  can  be  seen  that  Portfolio  A  remains  Pareto  optimal  through  all  epochs. 
Portfolio  C,  which  may  have  been  overlooked  in  traditional  analysis  as  it  does  not  appear 
Pareto  optimal  in  two  of  the  epochs,  is  the  only  other  design  which  is  Pareto  dominant  in  all 
epochs  when  a  10%  fuzziness  factor  is  applied.  Therefore,  Portfolio  C  represents  a  passive 
robust  solution  that  remains  affordable  against  uncertainty,  but  is  displaced  in  some  cases 
from  the  Pareto  frontier.  Portfolios  A  and  C  are  composed  of  identical  types  of  constituent 
systems,  simply  with  different  numbers  of  these  systems.  This  may  indicate  to  a  portfolio 
designer  that  the  constituent  systems  in  these  portfolios  provide  promising  value  to  potential 
future  CSGs  while  maintaining  affordability  under  potential  uncertainty. 

Finally,  though  single-era  analysis,  PLEEAA  discerned  which  initially  promising 
portfolio  designs  were  likely  to  remain  affordable,  while  maintaining  their  utility,  over  the 
lifecycle  of  the  CSG.  Figure  9  illustrates  that  while  all  portfolios  maintain  sufficient  utility 
delivery,  Portfolios  B,  D,  and  E  become  unaffordable  in  the  Small  Navy  epoch.  Upon 
investigation,  these  portfolios  were  found  to  exceed  the  operational  costs  allowable  under 
the  context  of  hypothetical  Navy  downsizing,  while  Portfolios  A  and  C  remained  affordable. 
Single-era  analysis  of  portfolios  provides  a  mechanism  to  identify  challenging  lifecycle 
circumstances  and  robust  solutions  during  conceptual  design. 

Discussion  of  PLEEAA  Application  to  Portfolio  Conceptual  Design  and 
Analysis 

The  conceptual  design  of  systems  of  systems  and  portfolios  presents  a  variety  of 
challenges  to  traditional  toolsets,  including  influence  considerations,  complexity  of 
combination  factors,  and  dynamically  changing  portfolio  composition.  However,  with  the 
prevalence  of  major  acquisition  Nunn-McCurdy  breeches  in  recent  years,  the  ability  to 
evaluate  the  affordability  of  portfolios  with  respect  to  lifecycle  uncertainty  is  desired  to 
support  acquisition  decision  makers.  The  PLEEAA  method  enables  designers  to  enumerate 
and  consider  a  greater  number  of  potential  portfolio  designs  than  would  be  possible  by 
current  techniques.  This  is  advantageous  because  it  increases  the  ability  of  a  designer  to 
identify  portfolios  which  display  superior  performance  qualities  such  as  affordability  and 
robustness  against  exogenous  uncertainty. 

Tradespace  exploration  supports  decision  makers  to  identify  portfolio  compositions 
they  may  not  have  previously  considered,  to  recognize  macro-level  trends  in  portfolio 
affordability,  and  to  conduct  micro-level  tradeoff  studies  between  portfolios  in  the  area  of 
desired  performance.  Epoch-Era  Analysis  provides  acquisitions  officers  with  new  abilities  to 
identify  how  potential  designs  may  respond  to  a  variety  of  anticipated  contextual 
uncertainties,  including  the  impact  of  simultaneously  occurring  uncertainties,  in  the  lifecycle 
development  and  operation  of  the  portfolio.  The  insight  provided  from  these  studies  will 
support  the  selection  of  portfolios  which  may  remain  affordable  over  the  entire  lifecycle  of 
the  program,  without  the  need  to  alter  requirements  or  design  in  response  to  changing 
contexts.  In  implementation,  the  authors  envision  a  network  of  system  designers,  each  using 
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tradespace  exploration  to  design  their  own  systems,  connected  by  a  SoS  and  interacting 
through  the  PLEEAA  method  to  ensure  that  needed  capabilities  are  provided  in  the  portfolio 
at  an  acceptable  cost  with  a  desired  level  of  uncertainty  robustness. 

Conclusion 

The  differences  between  capability  portfolios  of  assets  and  individual  systems 
necessitate  specialized  and  distinct  approaches  for  design  and  acquisition  planning.  Epoch- 
Era  Analysis  has  shown  promise  in  previous  studies  to  enable  the  design  of  affordable 
systems  that  provide  adequate  utility  while  remaining  under  cost  thresholds  through  a 
variety  of  potential  contexts  that  may  be  experienced  over  the  lifecycle  of  the  system. 

Modern  Portfolio  Theory  is  a  well-known  financial  tool  that  has  been  used  for  decades  to 
select  portfolios  of  investments  which  maximize  utility  subject  to  fixed  uncertainty.  This 
research  leveraged  EEA  with  elements  of  MPT  to  facilitate  design  for  affordability  of  systems 
of  systems  with  uncertain  futures  using  a  portfolio-based  hierarchical  perspective.  The 
proposed  method  was  used  to  explore  524,160  potential  carrier  strike  group  portfolio 
designs  across  five  different  epochs  and  two  eras.  This  type  of  analysis  may  help  decision 
makers  to  identify  long-term  acquisition  strategies  for  affordable  portfolios  that  are  resilient 
against  the  types  of  contextual  uncertainties  that  have  negatively  impacted  recent  DoD 
acquisition  efforts. 
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